Image Alignment Method and Apparatus

ABSTRACT

An image alignment method and apparatus, where the method and apparatus include obtaining image information of two to-be-aligned images, determining, using a cross-correlation measurement model, first coordinate offset according to the image information of the two images, where the first coordinate offset are used to indicate position deviations of to-be-aligned pixels between the two images in the coordinate system, and aligning the two images according to coordinates of pixels in the first image in the coordinate system and the first coordinate offset.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Patent ApplicationNo. PCT/CN2015/075823 filed on Apr. 3, 2015, which claims priority toChinese Patent Application No. 201410452309.4 filed on Sep. 5, 2014. Thedisclosures of the aforementioned applications are hereby incorporatedby reference in their entireties.

TECHNICAL FIELD

The present disclosure relates to the field of image processing, and inparticular, to an image alignment method and apparatus.

BACKGROUND

An image alignment technology is a foundational technology in the fieldof image processing. With rapid development of digital imaging, a seriesof applications based on the image alignment technology have emerged.These applications include generation of a panorama, generation of ahighly dynamic image, information fusion of two images, and the like.For example, a color image is restored using an infrared image, or ablur of another color image is removed using an image with noise.

With vigorous development of various image capture devices, how to alignmulti-modal multi-spectral images becomes a new problem, and thesemulti-modal multi-spectral images include a near infrared image, a colorimage, a depth image, a nuclear magnetic resonance image, an ultrasonicimage, and the like. Because of different capture devices and a dynamicnature of a capture scenario, there is a great difference in capturedimages. FIG. 1 shows four groups of common multi-modal multi-spectralimages, which are as follows from left to right. The first groupincludes images with different exposures, the second group includescolor and depth images, the third group includes color and near infraredimages, and the fourth group includes an image shot when a flash isenabled and an image shot when a flash is disabled. It can be learnedfrom FIG. 1 that main differences between the multi-modal multi-spectralimages are as follows, a large color contrast between the images, andlarge gradient value and gradient direction contrasts between theimages.

A conventional alignment technology based on a scale-invariant featuretransform (SIFT) feature point has been widely used in the imagealignment field. Further, in the technology, images that need to bealigned are matched by searching for the SIFT feature point. Two imagesare used as an example. According to an image alignment method based onthe SIFT feature point, SIFT feature point vectors of the two images arefirst extracted, and a nearest neighbor is found using a Euclideandistance between the vectors in order to obtain a correspondence betweenthe two images. However, the SIFT feature point is closely related to agradient value and a gradient direction that are between images, theimage alignment technology based on the SIFT feature point greatlydepends on gradient value and gradient direction consistencies that arebetween regions with similar image structures. However, it can belearned from FIG. 1 that, for multi-modal multi-spectral images, thereis a relatively large gradient direction contrast between the regionswith similar structures. Therefore, the image alignment technology basedon the SIFT feature point is not suitable for alignment between themulti-modal multi-spectral images.

SUMMARY

Embodiments of the present disclosure provide an image alignment methodand apparatus in order to improve image alignment accuracy.

According to a first aspect, an image alignment method is provided,including obtaining image information of two to-be-aligned images, whereimage information of a first image includes coordinates of pixels in thefirst image in a selected coordinate system, pixel values of the pixelsin the first image, and a pixel value gradient of the pixels in thefirst image, and image information of a second image includes pixelvalues of pixels in the second image and a pixel value gradient of thepixels in the second image, where the two images are located in thecoordinate system, determining, using a cross-correlation measurementmodel, first coordinate offset according to the image information of thetwo images, where the first coordinate offset are used to indicateposition deviations of to-be-aligned pixels between the two images inthe coordinate system, and aligning the two images according to thecoordinates of the pixels in the first image in the coordinate systemand the first coordinate offset.

With reference to the first aspect, in one implementation manner of thefirst aspect, before determining, using a cross-correlation measurementmodel, first coordinate offset according to the image information of thetwo images, the method further includes obtaining image information of athird image, where the image information of the third image includespixel values of pixels in the third image and a pixel value gradient ofthe pixels in the third image, the third image is located in thecoordinate system, and the first image and the third image areto-be-aligned original images, determining, using the cross-correlationmeasurement model, a coordinate transformation matrix according to imageinformation of the original images, where the coordinate transformationmatrix is used to indicate a spatial position relationship ofto-be-aligned pixels between the original images in the coordinatesystem, determining second coordinate offset according to the coordinatetransformation matrix, where the second coordinate offset are used toindicate position deviations of the to-be-aligned pixels between theoriginal images in the coordinate system, and obtaining the second imageaccording to the second coordinate offset and the pixel values of thepixels in the third image.

With reference to any one of the first aspect, or the foregoingimplementation manners of the first aspect, in another implementationmanner of the first aspect, determining, using the cross-correlationmeasurement model, a coordinate transformation matrix according to imageinformation of the original images includes determining the coordinatetransformation matrix by calculating a minimum value of

${{E_{1}(H)} = {\sum\limits_{p}{E_{2}\left( {p,w_{p}} \right)}}},$

where E₂(p,w_(p))=ρ(1−|Φ_(I)(p,w_(p))|)+τρ(1−|Φ_(∇I)(p,w_(p))|),

${{\Phi_{I}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{3,p} - I_{3,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{3,p} - I_{3,p}^{\prime}}}}},{{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}}}}},{{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$

p=(x_(p),y_(p))^(T), and w_(p)=(u_(p),v_(p))^(T), H indicates thecoordinate transformation matrix, and H meets[u_(p),v_(p),1]^(T)=[x_(p),y_(p),1]^(T)(H−I), I indicates an identitymatrix, p indicates the coordinates of the pixels in the first image inthe coordinate system, x_(p) indicates a horizontal coordinate of p,y_(p) indicates a vertical coordinate of p, w_(p) indicates the secondcoordinate offset, u_(p) indicates a horizontal coordinate of w_(p),v_(p) indicates a vertical coordinate of w_(p), I_(1,p) indicates aone-dimensional column vector including pixel values of pixels in ap-centered image block of the first image, ∇I_(1,p) indicates aone-dimensional column vector including pixel value gradients of thepixels in the image block, I′_(1,p) indicates a one-dimensional columnvector including pixel means of the pixels in the image block, ∇I′_(1,p)indicates a one-dimensional column vector including pixel value gradientmeans of the pixels in the image block, I_(3,p) indicates aone-dimensional column vector including pixel values of pixels in a(p+w_(p))-centered image block of the third image, ∇I_(3,p) indicates aone-dimensional column vector including pixel value gradients of thepixels in the (p+w_(p))-centered image block, I′_(3,p) indicates aone-dimensional column vector including pixel means of the pixels in the(p+w_(p))-centered image block, ∇I′_(3,p) indicates a one-dimensionalcolumn vector including pixel value gradient means of the pixels in the(p+w_(p))-centered image block, and β and τ are constants, where β isused to control a shape of a function ρ(x), and τ is a weight ofρ(1−|Φ_(∇I)(p,w_(p))| in E₂(p,w_(p))ρ(x).

With reference to any one of the first aspect, or the foregoingimplementation manners of the first aspect, in another implementationmanner of the first aspect, the determining second coordinate offsetaccording to the coordinate transformation matrix includes determiningthe second coordinate offset according to a formula[u_(p),v_(p),1]^(T)=[x_(p),y_(p),1]^(T)(H−I).

With reference to any one of the first aspect, or the foregoingimplementation manners of the first aspect, in another implementationmanner of the first aspect, obtaining the second image according to thesecond coordinate offset and the pixel values of the pixels in the thirdimage includes obtaining the second image according to a formulaI₂(p)=I₃(p+w_(p)), where I₂(p) indicates a pixel value of the secondimage in p, and I₃(p+w_(p)) indicates a pixel value of the third imagein p+w_(p).

With reference to any one of the first aspect, or the foregoingimplementation manners of the first aspect, in another implementationmanner of the first aspect, determining, using a cross-correlationmeasurement model, first coordinate offset according to the imageinformation of the two images includes determining the first coordinateoffset according to a formula

${{E_{3}\left( w_{p}^{\prime} \right)} = {{\sum\limits_{p}\; {E_{2}\left( {p,w_{p}^{\prime}} \right)}} + {\lambda_{1}{\sum\limits_{p}\; {\psi \left( {{\nabla w_{p}^{\prime}}}^{2} \right)}}} + {\lambda_{2}{\sum\limits_{p}\; {\sum\limits_{q \in {N{(p)}}}{{w_{p}^{\prime} - w_{q}^{\prime}}}}}}}},$

where E₂(p,w′_(p))=ρ(1−|Φ_(I)(p,w′_(p))|)+τρ(1−|Φ_(∇I)(p,w′_(p))|),

$\begin{matrix}{{{\Phi_{1}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},} \\{{{\Phi_{\nabla I}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},}\end{matrix}$

p=(x_(p),y_(p))^(T), w′_(p)=(u′_(p),v′_(p))^(T),

${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$

and ψ(x²)=√{square root over (x²+ε²)}, p indicates the coordinates ofthe pixels in the first image in the coordinate system, x_(p) indicatesthe horizontal coordinate of p, y_(p) indicates the vertical coordinateof p, w′_(p) indicates the first coordinate offset, u′_(p) indicates ahorizontal coordinate of w′_(p), v′_(p) indicates a vertical coordinateof w′_(p), I_(1,p) indicates the one-dimensional column vector includingthe pixel values of the pixels in the p-centered image block of thefirst image, ∇I_(1,p) indicates the one-dimensional column vectorincluding the pixel value gradients of the pixels in the image block,I′_(1,p) indicates the one-dimensional column vector including the pixelmeans of the pixels in the image block, ∇I′_(1,p) indicates theone-dimensional column vector including the pixel value gradient meansof the pixels in the image block, I_(2,p) indicates a one-dimensionalcolumn vector including pixel values of pixels in a (p+w′_(p))-centeredimage block of the second image, ∇I_(2,p) indicates a one-dimensionalcolumn vector including pixel value gradients of the pixels in the(p+w′_(p))-centered image block, I′_(2,p) indicates a one-dimensionalcolumn vector including pixel means of the pixels in the(p+w′_(p))-centered image block, ∇I′_(2,p) indicates a one-dimensionalcolumn vector including pixel value gradient means of the pixels in the(p+w′_(p))-centered image block, λ₁, β, λ₂ and τ are constants, where λ₁and λ₂ are weights of the second term and the third term that are inE₃(w′_(p)), β is used to control the shape of the function ρ(x), and τis the weight of ρ(1−|Φ_(∇I)(p,w′_(p))| in E₂(p,w′_(p)), N(p) indicatesa set including adjacent pixels of a pixel p in the first image, qindicates any pixel in the set, w_(q) indicates a coordinate offsetbetween q and a to-be-aligned pixel of q in the second image, and ε is aconstant.

With reference to any one of the first aspect, or the foregoingimplementation manners of the first aspect, in another implementationmanner of the first aspect, the cross-correlation measurement model isas follows E₂(p,w_(p))=ρ(1−|Φ₁(p,w_(p))|)+τρ(1−|Φ_(∇I)(p,w_(p))|), where

$\begin{matrix}{{{\Phi_{1}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},} \\{{{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},}\end{matrix}$

p=(x_(p),y_(p))^(T), w_(p)=(u_(p),v_(p))^(t), and

${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$

p indicates the coordinates of the pixels in the first image in thecoordinate system, x_(p) indicates the horizontal coordinate of p, y_(p)indicates the vertical coordinate of p, w_(p) indicates the firstcoordinate offset, u_(p) indicates the horizontal coordinate of w_(p),v_(p) indicates the vertical coordinate of w_(p), I_(1,p) indicates theone-dimensional column vector including the pixel values of the pixelsin the p-centered image block of the first image, ∇I_(1,p) indicates theone-dimensional column vector including the pixel value gradients of thepixels in the image block, I′_(1,p) indicates the one-dimensional columnvector including the pixel means of the pixels in the image block,∇I′_(1,p) indicates the one-dimensional column vector including thepixel value gradient means of the pixels in the image block, I_(2,p)indicates a one-dimensional column vector including pixel values ofpixels in a (p+w_(p))-centered image block of the second image, ∇I_(2,p)indicates a one-dimensional column vector including pixel valuegradients of the pixels in the (p+w_(p))-centered image block, I′_(2,p)indicates a one-dimensional column vector including pixel means of thepixels in the (p+w_(p))-centered image block, ∇I_(2,p) indicates aone-dimensional column vector including pixel value gradient means ofthe pixels in the (p+w_(p))-centered image block, and β is a weight andis used to control the shape of the function ρ(x).

According to a second aspect, an image alignment apparatus is provided,including a first obtaining unit configured to obtain image informationof two to-be-aligned images, where image information of a first imageincludes coordinates of pixels in the first image in a selectedcoordinate system, pixel values of the pixels in the first image, and apixel value gradient of the pixels in the first image, and imageinformation of a second image includes pixel values of pixels in thesecond image and a pixel value gradient of the pixels in the secondimage, where the two images are located in the coordinate system, afirst determining unit configured to determine, using across-correlation measurement model, first coordinate offset accordingto the image information of the two images, where the first coordinateoffset are used to indicate position deviations of to-be-aligned pixelsbetween the two images in the coordinate system, and an alignment unitconfigured to align the two images according to the coordinates of thepixels in the first image in the coordinate system and the firstcoordinate offset.

With reference to the second aspect, in one implementation manner of thesecond aspect, the apparatus further includes a second obtaining unitconfigured to obtain image information of a third image, where the imageinformation of the third image includes pixel values of pixels in thethird image and a pixel value gradient of the pixels in the third image,the third image is located in the coordinate system, and the first imageand the third image are to-be-aligned original images, a seconddetermining unit configured to determine, using the cross-correlationmeasurement model, a coordinate transformation matrix according to imageinformation of the original images, where the coordinate transformationmatrix is used to indicate a spatial position relationship ofto-be-aligned pixels between the original images in the coordinatesystem, a third determining unit configured to determine secondcoordinate offset according to the coordinate transformation matrix,where the second coordinate offset are used to indicate positiondeviations of the to-be-aligned pixels between the original images inthe coordinate system, and a fourth determining unit configured toobtain the second image according to the second coordinate offset andthe pixel values of the pixels in the third image.

With reference to any one of the second aspect, or the foregoingimplementation manners of the second aspect, in another implementationmanner of the second aspect, the second determining unit is furtherconfigured to determine the coordinate transformation matrix bycalculating a minimum value of

${{E_{1}(H)} = {\sum\limits_{p}{E_{2}\left( {p,w_{p}} \right)}}},$

where E₂(p,w_(p))=ρ(1−|Φ_(I)(p,w_(p))|)+τρ(1−|Φ_(∇I)(p,w_(p))|),

$\begin{matrix}{{{\Phi_{1}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{3,p} - I_{3,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{3,p} - I_{3,p}^{\prime}}}}},} \\{{{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}}}}},} \\{{{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},}\end{matrix}$

p=(x_(p),y_(p))^(T), and w_(p)=(u_(p),v_(p))^(T), H indicates thecoordinate transformation matrix, and H meets[u_(p),v_(p),1]^(T)=[x_(p),y_(p),1]^(T)(H−I), I indicates an identitymatrix, P indicates the coordinates of the pixels in the first image inthe coordinate system, x_(p) indicates a horizontal coordinate of p,y_(p) indicates a vertical coordinate of p, w_(p) indicates the secondcoordinate offset, u_(p) indicates a horizontal coordinate of w_(p),v_(p) indicates a vertical coordinate of w_(p), I_(1,p) indicates aone-dimensional column vector including pixel values of pixels in ap-centered image block of the first image, ∇I_(1,p) indicates aone-dimensional column vector including pixel value gradients of thepixels in the image block, I′_(1,p) indicates a one-dimensional columnvector including pixel means of the pixels in the image block, ∇I′_(1,p)indicates a one-dimensional column vector including pixel value gradientmeans of the pixels in the image block, I_(3,p) indicates aone-dimensional column vector including pixel values of pixels in a(p+w_(p))-centered image block of the third image, ∇I_(3,p) indicates aone-dimensional column vector including pixel value gradients of thepixels in the (p+w_(p))-centered image block, I′_(3,p) indicates aone-dimensional column vector including pixel means of the pixels in the(p+w_(p))-centered image block, ∇I′_(3,p) indicates a one-dimensionalcolumn vector including pixel value gradient means of the pixels in the(p+w_(p))-centered image block, and β and τ are constants, where β isused to control a shape of a function ρ(x), and τ is a weight ofρ(1−|Φ_(∇I)(p,w_(p))| in E₂(p,w_(p))ρ(x).

With reference to any one of the second aspect, or the foregoingimplementation manners of the second aspect, in another implementationmanner of the second aspect, the third determining unit is furtherconfigured to determine the second coordinate offset according to aformula [u_(p),v_(p),1]^(T)=[x_(p),y_(p),1]^(T)(H−I).

With reference to any one of the second aspect, or the foregoingimplementation manners of the second aspect, in another implementationmanner of the second aspect, the fourth determining unit is furtherconfigured to obtain the second image according to a formulaI₂(p)=I₃(p+w_(p)), where I₂(p) indicates a pixel value of the secondimage in p, and I₃(p+w_(p)) indicates a pixel value of the third imagein p+w_(p).

With reference to any one of the second aspect, or the foregoingimplementation manners of the second aspect, in another implementationmanner of the second aspect, the first determining unit is furtherconfigured to determine the first coordinate offset according to aformula

${{E_{3}\left( w_{p}^{\prime} \right)} = {{\sum\limits_{p}\; {E_{2}\left( {p,w_{p}^{\prime}} \right)}} + {\lambda_{1}{\sum\limits_{p}\; {\psi \left( {{\nabla w_{p}^{\prime}}}^{2} \right)}}} + {\lambda_{2}{\sum\limits_{p}\; {\sum\limits_{q \in {N{(p)}}}{{w_{p}^{\prime} - w_{q}^{\prime}}}}}}}},$

where

$\begin{matrix}{{{E_{2}\left( {p,w_{p}^{\prime}} \right)} = {{\rho \left( {1 - {{\Phi_{I}\left( {p,w_{p}^{\prime}} \right)}}} \right)} + {\tau \; \rho \; \left( {1 - {{\Phi_{\nabla I}\left( {p,w_{p}^{\prime}} \right)}}} \right)}}},} \\{{{\Phi_{1}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},} \\{{{\Phi_{\nabla I}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},}\end{matrix}$

p=(x_(p),y_(p))^(T), w′_(p)=(u′_(p),v′_(p))^(T),

${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$

and ψ(x²)=√{square root over (x²+ε²)}, p indicates the coordinates ofthe pixels in the first image in the coordinate system, x_(p) indicatesthe horizontal coordinate of p, y_(p) indicates the vertical coordinateof p, w′_(p) indicates the first coordinate offset, u′_(p) indicates ahorizontal coordinate of w′_(p), v′_(p) indicates a vertical coordinateof w′_(p), I_(1,p) indicates the one-dimensional column vector includingthe pixel values of the pixels in the p-centered image block of thefirst image, ∇I_(1,p) indicates the one-dimensional column vectorincluding the pixel value gradients of the pixels in the image block,I′_(1,p) indicates the one-dimensional column vector including the pixelmeans of the pixels in the image block, ∇I′_(1,p) indicates theone-dimensional column vector including the pixel value gradient meansof the pixels in the image block, I_(2,p) indicates a one-dimensionalcolumn vector including pixel values of pixels in a (p+w′_(p))-centeredimage block of the second image, ∇I_(2,p) indicates a one-dimensionalcolumn vector including pixel value gradients of the pixels in the(p+w′_(p))-centered image block, I′_(2,p) indicates a one-dimensionalcolumn vector including pixel means of the pixels in the(p+w′_(p))-centered image block, ∇I_(2,p) indicates a one-dimensionalcolumn vector including pixel value gradient means of the pixels in the(p+w_(p))-centered image block λ₁, β, λ₂ and τ are constants, where λ₁and λ₂ are weights of the second term and the third term that are inE₃(w′_(p)), β is used to control the shape of the function ρ(x), and τis the weight of ρ(1−|Φ_(∇I)(p,w′_(p))| in E₂(p,w′_(p)), N(p) indicatesa set including adjacent pixels of a pixel p in the first image, qindicates any pixel in the set, w_(q) indicates a coordinate offsetbetween q and a to-be-aligned pixel of q in the second image, and ε is aconstant.

With reference to any one of the second aspect, or the foregoingimplementation manners of the second aspect, in another implementationmanner of the second aspect, the cross-correlation measurement model isas follows, E₂(p,w_(p))=ρ(1−|Φ_(I)(p,w_(p))|)+τρ(1−|Φ_(∇I)(p,w_(p))|),where

$\begin{matrix}{{{\Phi_{1}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},} \\{{{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},}\end{matrix}$

p=(x_(p),y_(p))^(T), w_(p)=(u_(p),v_(p))^(T), and

${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$

p indicates the coordinates of the pixels in the first image in thecoordinate system, x_(p) indicates the horizontal coordinate of p, y_(p)indicates the vertical coordinate of p, w_(p) indicates the firstcoordinate offset, u_(p) indicates the horizontal coordinate of w_(p),v_(p) indicates the vertical coordinate of w_(p), I_(1,p) indicates theone-dimensional column vector including the pixel values of the pixelsin the p-centered image block of the first image, ∇I_(1,p) indicates theone-dimensional column vector including the pixel value gradients of thepixels in the image block, I′_(1,p) indicates the one-dimensional columnvector including the pixel means of the pixels in the image block,∇I′_(1,p) indicates the one-dimensional column vector including thepixel value gradient means of the pixels in the image block, I_(2,p)indicates a one-dimensional column vector including pixel values ofpixels in a (p+w_(p))-centered image block of the second image, ∇I_(2,p)indicates a one-dimensional column vector including pixel valuegradients of the pixels in the (p+w_(p))-centered image block, I′^(2,p)indicates a one-dimensional column vector including pixel means of thepixels in the (p+w_(p))-centered image block, ∇I′_(2,p) indicates aone-dimensional column vector including pixel value gradient means ofthe pixels in the (p+w_(p))-centered image block, and β is a weight andis used to control the shape of the function ρ(x).

According to the embodiments of the present disclosure, thecross-correlation measurement model is introduced. Because both a colorcross-correlation and a gradient cross-correlation that are betweenimages are considered in the cross-correlation measurement model,compared with an existing image alignment technology based on a SIFTfeature point, the cross-correlation measurement model is more suitablefor alignment between multi-modal multi-spectral images and improvesimage alignment accuracy.

BRIEF DESCRIPTION OF DRAWINGS

To describe the technical solutions in the embodiments of the presentdisclosure more clearly, the following briefly describes theaccompanying drawings required for describing the embodiments of thepresent disclosure. The accompanying drawings in the followingdescription show merely some embodiments of the present disclosure, anda person of ordinary skill in the art may still derive other drawingsfrom these accompanying drawings without creative efforts.

FIG. 1 is an example diagram of multi-modal multi-spectral images;

FIG. 2A is a function curve diagram of a robust function;

FIG. 2B is a function curve diagram of the derivative function of therobust function of FIG. 2A;

FIG. 3 is a schematic flowchart of an image alignment method accordingto an embodiment of the present disclosure;

FIG. 4 is a schematic block diagram of an image alignment apparatusaccording to an embodiment of the present disclosure; and

FIG. 5 is a schematic block diagram of an image alignment apparatusaccording to an embodiment of the present disclosure.

DESCRIPTION OF EMBODIMENTS

The following clearly describes the technical solutions in theembodiments of the present disclosure with reference to the accompanyingdrawings in the embodiments of the present disclosure. The describedembodiments are a part rather than all of the embodiments of the presentdisclosure. All other embodiments obtained by a person of ordinary skillin the art based on the embodiments of the present disclosure withoutcreative efforts shall fall within the protection scope of the presentdisclosure.

In the image alignment field, selection of a structural similaritymetric between images is directly related to a final image alignmenteffect. In the prior art, structural similarity metrics include a color,a gradient, a SIFT feature point, cross-correlation information, and thelike. However, because there are relatively large color and gradientdirection contrasts between multi-modal multi-spectral images, it isdifficult to accurately describe structural similarity between themulti-modal multi-spectral images using any one of the above structuralsimilarity metrics.

According to an embodiment of the present disclosure, across-correlation measurement (which may also be referred to as a robustselective standardized cross-correlation measurement) model isintroduced to describe structural similarity between images. Thecross-correlation measurement model is established based on a colorcross-correlation and a gradient cross-correlation that are between theimages, or the cross-correlation measurement model is used to indicate acolor cross-correlation and a gradient cross-correlation that arebetween the images. It is assumed that to-be-aligned images include afirst image and a second image, the first image is represented by I₁,and the second image is represented by I₂. The cross-correlationmeasurement model may be indicated as a formula

E₂(p, w_(p)) = ρ(1 − Φ_(I)(p, w_(p))) + τ ρ (1 − Φ_(∇I)(p, w_(p))),

where

${{\Phi_{I}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},{{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},$

p=(x_(p),y_(p))^(T), w_(p)=(u_(p),v_(p))^(T), and

${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$

p indicates coordinates of pixels in the first image, and thecoordinates may be any one of preselected coordinates, provided that theto-be-aligned images are located in a coordinate system, x_(p) indicatesa horizontal coordinate of p, y_(p) indicates a vertical coordinate ofp, w_(p) indicates a first coordinate offset, u_(p) indicates ahorizontal coordinate of w_(p), v_(p) indicates a vertical coordinate ofw_(p), I_(1,p) indicates a one-dimensional column vector including pixelvalues of pixels in a p-centered image block of the first image,∇I_(1,p) indicates a one-dimensional column vector including pixel valuegradients of the pixels in the image block, I′_(1,p) indicates aone-dimensional column vector including pixel means of the pixels in theimage block, ∇I′_(1,p) indicates a one-dimensional column vectorincluding pixel value gradient means of the pixels in the image block,I_(2,p) indicates a one-dimensional column vector including pixel valuesof pixels in a (p+w_(p))-centered image block of the second image,∇I_(2,p) indicates a one-dimensional column vector including pixel valuegradients of the pixels in the (p+w_(p))-centered image block, I′_(2,p)indicates a one-dimensional column vector including pixel means of thepixels in the (p+w_(p))-centered image block, ∇I′_(2,p) indicates aone-dimensional column vector including pixel value gradient means ofthe pixels in the (p+w_(p))-centered image block, and β is a weight andis used to control a shape of a function ρ(x).

In the foregoing formula, Φ₁(p,w_(p)) is a standardizedcross-correlation function, and is used to calculate cross-correlationbetween color values of the p-centered image block in the first image I₁and the (p+w_(p))-centered image block in the second image I₂. Likewise,Φ_(∇I)(p,w_(p)) may also be a standardized cross-correlation function,and is used to calculate cross-correlation between gradient values ofthe p-centered image block in the first image I₁ and the(p+w_(p))-centered image block in the second image I₂.

In addition, the robust function

${\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}$

is introduced in a cross-correlation model, and the robust function is aconsecutive derivative function. For a function image of the robustfunction, refer to FIGS. 2A and 2B. FIG. 2A is a function curve of therobust function, and FIG. 2B is a function curve of the derivativefunction of the robust function. It can be learned from FIGS. 2A and 2Bthat the robust function is an increasing function of x. However, withincreasing of x, a growth speed of the robust function becomes lower,thereby achieving a great robust effect.

The following describes in detail an image alignment method in anembodiment of the present disclosure with reference to FIG. 3.

FIG. 3 is a schematic flowchart of the image alignment method accordingto this embodiment of the present disclosure. The method shown in FIG. 3includes the following steps.

Step 310: Obtain image information of two to-be-aligned images, whereimage information of a first image includes coordinates of pixels in thefirst image in a selected coordinate system, pixel values of the pixelsin the first image, and a pixel value gradient of the pixels in thefirst image, and image information of a second image includes pixelvalues of pixels in the second image and a pixel value gradient of thepixels in the second image, where the two images are located in thecoordinate system.

Step 320: Determine, using a cross-correlation measurement model, firstcoordinate offset according to the image information of the two images,where the first coordinate offset are used to indicate positiondeviations of to-be-aligned pixels between the two images in thecoordinate system.

It should be noted that it is assumed that both the first image and thesecond image include N pixels. The foregoing pixels in the first imagemay refer to the N pixels in the first image, and the foregoing pixelsin the second image may refer to the N pixels in the second image.Conceptually, the foregoing first coordinate offset may be a set.Further, all the N pixels in the first image may be corresponding to Nfirst coordinate offset.

Step 330: Align the two images according to the coordinates of thepixels in the first image in the coordinate system and the firstcoordinate offset.

For example, a coordinate point of a pixel in the first image is p, andthe first coordinate offset corresponding to p is w_(p). Step 330 mayinclude, first finding a pixel value x of a coordinate point p+w_(p) inthe second image, and then updating a pixel value y of a coordinatepoint p in the first image with the pixel value x.

It can be learned from the foregoing that multi-modal multi-spectralimages feature large color, gradient value, and gradient directioncontrasts. According to this embodiment of the present disclosure, thecross-correlation measurement model is introduced. Because both a colorcross-correlation and a gradient cross-correlation that are betweenimages are considered in the cross-correlation measurement model,compared with an existing image alignment technology based on a SIFTfeature point, the cross-correlation measurement model is more suitablefor alignment between multi-modal multi-spectral images and improvesimage alignment accuracy.

Optionally, in an embodiment, before step 320, the method shown in FIG.3 may further include obtaining image information of a third image,where the image information of the third image includes pixel values ofpixels in the third image and a pixel value gradient of the pixels inthe third image, the third image is located in the coordinate system,and the first image and the third image are to-be-aligned originalimages, determining, using the cross-correlation measurement model, acoordinate transformation matrix according to image information of theoriginal images, where the coordinate transformation matrix is used toindicate a spatial position relationship of to-be-aligned pixels betweenthe original images in the coordinate system, determining secondcoordinate offset according to the coordinate transformation matrix,where the second coordinate offset are used to indicate positiondeviations of the to-be-aligned pixels between the original images inthe coordinate system, and obtaining the second image according to thesecond coordinate offset and the pixel values of the pixels in the thirdimage.

It may be understood that the first image and the third image are theto-be-aligned original images. In this embodiment, an alignment betweenthe first image and the third image is decomposed into two alignments. Aglobal alignment is first performed, and then a pixel-level alignment isperformed. After the global alignment, the third image is updated withthe second image. The second image is an intermediate image generated ina process of the alignment between the first image and the third image.

It should be noted that the foregoing global alignment between theimages is implemented by solving the coordinate transformation matrix.The foregoing coordinate transformation matrix may be a homographymatrix, and the coordinate transformation matrix may be used to describean appropriate alignment between the third image and the first image bymeans of global translation, rotation, and zooming.

The following gives an embodiment of a process of solving the coordinatetransformation matrix.

The coordinate transformation matrix may be solved by calculating aminimum value of an energy function

${{E_{1}(H)} = {\sum\limits_{p}{E_{2}\left( {p,w_{p}} \right)}}},$

where E₂(p,w_(p))=ρ(1−|Φ₁(p,w_(p))|)+τρ(1−|Φ_(∇I)(p,w_(p))|),

$\begin{matrix}{{{\Phi_{1}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{3,p} - I_{3,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{3,p} - I_{3,p}^{\prime}}}}},} \\{{{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}}}}},} \\{{{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},}\end{matrix}$

p=(x_(p),y_(p))^(T), and w_(p)=(u_(p),v_(p))^(T), H indicates thecoordinate transformation matrix, and H meets[u_(p),v_(p)1,]^(T)=[x_(p),y_(p),1]^(T)(H−I), I indicates an identitymatrix, p indicates the coordinates of the pixels in the first image inthe coordinate system, x_(p) indicates a horizontal coordinate of p,y_(p) indicates a vertical coordinate of p, w_(p) indicates the secondcoordinate offset, u_(p) indicates a horizontal coordinate of w_(p),v_(p) indicates a vertical coordinate of w_(p), I_(1,p) indicates aone-dimensional column vector including pixel values of pixels in ap-centered image block of the first image, ∇I_(1,p) indicates aone-dimensional column vector including pixel value gradients of thepixels in the image block, I′_(1,p) indicates a one-dimensional columnvector including pixel means of the pixels in the image block, ∇I′_(1,p)indicates a one-dimensional column vector including pixel value gradientmeans of the pixels in the image block, I_(3,p) indicates aone-dimensional column vector including pixel values of pixels in a(p+w_(p))-centered image block of the third image, ∇I_(3,p) indicates aone-dimensional column vector including pixel value gradients of thepixels in the (p+w_(p))-centered image block, I′_(3,p) indicates aone-dimensional column vector including pixel means of the pixels in the(p+w_(p))-centered image block, ∇I′_(3,p) indicates a one-dimensionalcolumn vector including pixel value gradient means of the pixels in the(p+w_(p))-centered image block, and β and τ are constants, where β isused to control a shape of a function ρ(x), and τ is a weight ofρ(1−|Φ_(∇I)(p,w_(p))| in E₂(p,w_(p)).

It should be noted that in

${{E_{1}(H)} = {\sum\limits_{p}{E_{2}\left( {p,w_{p}} \right)}}},$

because H meets [u_(p),v_(p),1]^(T)=[x_(p),y_(p),1]^(T)(H−I), w_(p) isfirst replaced with H, and then the minimum value of the equation may besolved using a steepest descent method. To further improve efficiency ofan algorithm, H may be gradually optimized using an image pyramidmethod. Further, solving is quickly performed first from a bottom layer(the bottom layer has a lowest resolution) of an image pyramid, then aresult is transmitted to an upper layer for further optimization, andfinally optimization is performed from a top layer that is of thepyramid and that has a highest resolution. A running speed of thealgorithm can be significantly improved using this optimization policy.

After the coordinate transformation matrix is obtained, the determiningsecond coordinate offset according to the coordinate transformationmatrix may include determining the second coordinate offset according toa formula [u_(p),v_(p),1]^(T)=[x_(p),y _(p),1]^(T)(H−I).

Further, after H is determined, the second coordinate offset(u_(p),v_(p)) corresponding to pixels are calculated by substituting thecoordinates (x_(p),y_(p)) of the pixels in the first image into theabove formula.

After the second coordinate offset are obtained, the second image isobtained according to a formula I₂(p)=I₃(p+w_(p)), where I₂(p) indicatesa pixel value of the second image in p, and I₃(p+w_(p)) indicates apixel value of the third image in p+w_(p). In this case, the foregoingsecond image is obtained.

After the global alignment between the images is completed, thepixel-level alignment may be performed. Further, the first coordinateoffset may be determined according to a formula

${{E_{3}\left( w_{p}^{\prime} \right)} = {{\sum\limits_{p}{E_{2}\left( {\rho,w_{p}^{\prime}} \right)}} + {\lambda_{1}{\sum\limits_{p}{\Psi \left( {{\nabla w_{p}^{\prime}}}^{2} \right)}}} + {\lambda_{2}{\sum\limits_{p}{\sum\limits_{q \in {N{(p)}}}{{w_{p}^{\prime} - w_{q}^{\prime}}}}}}}},$

where E₂(p,w′_(p))=ρ(1−|Φ₁(p,w′_(p))|)+τρ(1−|Φ_(∇I)(p,w′_(p))|),

$\begin{matrix}{{{\Phi_{1}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},} \\{{{\Phi_{\nabla I}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},}\end{matrix}$

p=(x_(p),y_(p))^(T), w′_(p)=(u′_(p),v′_(p))^(T),

${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$

and ψ(x²)=√{square root over (x²+ε²)}, p indicates the coordinates ofthe pixels in the first image in the coordinate system, x_(p) indicatesthe horizontal coordinate of p, y_(p) indicates the vertical coordinateof p, w′_(p) indicates the first coordinate offset, u′_(p) indicates ahorizontal coordinate of w′_(p), v′_(p) indicates a vertical coordinateof w′_(p), I_(1,p) indicates the one-dimensional column vector includingthe pixel values of the pixels in the p-centered image block of thefirst image, ∇I_(1,p) indicates the one-dimensional column vectorincluding the pixel value gradients of the pixels in the image block,I′_(1,p) indicates the one-dimensional column vector including the pixelmeans of the pixels in the image block, ∇I′_(1,p) indicates theone-dimensional column vector including the pixel value gradient meansof the pixels in the image block, I_(2,p) indicates a one-dimensionalcolumn vector including pixel values of pixels in a (p+w_(p))-centeredimage block of the second image, ∇I_(2,p) indicates a one-dimensionalcolumn vector including pixel value gradients of the pixels in the(p+w_(p))-centered image block, I′_(2,p) indicates a one-dimensionalcolumn vector including pixel means of the pixels in the(p+w_(p))-centered image block, ∇I′_(2,p) indicates a one-dimensionalcolumn vector including pixel value gradient means of the pixels in the(p+w′_(p))-centered image block, λ₁, β, λ₂ and τ are constants, where λ₁and λ₂ are weights of the second term and the third term that are inE₃(w′_(p)), β is used to control the shape of the function ρ(x), and τis the weight of ρ(1−|Φ_(∇I)(p,w′_(p))| in E₂(p,w′_(p)), N(p) indicatesa set including adjacent pixels of a pixel p in the first image, qindicates any pixel in the set, w_(q) indicates a coordinate offsetbetween q and a to-be-aligned pixel of q in the second image, and ε is aconstant.

Further, the alignment between the first image and the second image maybe performed using the first term

$\sum\limits_{p}{E_{2}\left( {p,w_{p}^{\prime}} \right)}$

in

${E_{3}\left( w_{p}^{\prime} \right)} = {{\sum\limits_{p}{E_{2}\left( {\rho,w_{p}^{\prime}} \right)}} + {\lambda_{1}{\sum\limits_{p}{\psi \left( {{\nabla w_{p}^{\prime}}}^{2} \right)}}} + {\lambda_{2}{\sum\limits_{p}{\sum\limits_{q \in {N{(p)}}}{{w_{p}^{\prime} - w_{q}^{\prime}}}}}}}$

and under a constraint of the cross-correlation measurement model. Thesecond term

$\lambda_{1}{\sum\limits_{p}{\psi \left( {{\nabla w_{p}^{\prime}}}^{2} \right)}}$

and the third term

$\lambda_{2}{\sum\limits_{p}{\sum\limits_{q \in {N{(p)}}}{{w_{p}^{\prime} - w_{q}^{\prime}}}}}$

are regular terms, and smoothness of the offsets w′_(p) between pixelscan be ensured using the second term and the third term. Piecewisesmoothness of the offsets w′_(p) between the pixels can be ensured usingthe second term, where ψ(x²)=√{square root over (x²+ε²)} is a robustfunction, and ε is a relatively small constant and may be 1E−4. Thethird term is a median filtering term, and is used to improve smoothnessof w′_(p) and improve accuracy of solving w′_(p).

In addition, how to solve the energy function is a variational problem,and solving may be performed using an Euler-Lagrange equation. Toaccelerate the solving, a process of solving from a rough layer to afine layer may be used. Further, Gaussian pyramids of the first imageand the second image may be first established, and solving starting froma roughest layer of each of the Gaussian pyramids. Then, a result of arough layer is transmitted to a fine layer, and is used as an initialvalue of the fine layer for further solving until the result istransmitted to a finest layer (i.e. an original image). The runningspeed of the algorithm can be significantly improved, and a relativelylarge w′_(p) can be accurately solved using a solving policy.

It should be understood that, in various embodiments of the presentdisclosure, sequence numbers of the foregoing processes do not meanexecution sequences. The execution sequences of the processes should bedetermined according to functions and internal logic thereof, and shouldnot be construed as any limitation on the implementation processes ofthe embodiments of the present disclosure.

The foregoing describes in detail the image alignment method accordingto the embodiments of the present disclosure with reference to FIG. 1 toFIG. 3. The following describes in detail image alignment apparatusesaccording to the embodiments of the present disclosure with reference toFIG. 4 and FIG. 5.

It should be understood that the apparatuses in FIG. 4 and FIG. 5 canimplement steps in FIG. 3. To avoid repetition, details are notdescribed herein again.

FIG. 4 is a schematic block diagram of an image alignment apparatusaccording to an embodiment of the present disclosure. The apparatus 400in FIG. 4 includes a first obtaining unit 410 configured to obtain imageinformation of two to-be-aligned images, where image information of afirst image includes coordinates of pixels in the first image in aselected coordinate system, pixel values of the pixels in the firstimage, and a pixel value gradient of the pixels in the first image, andimage information of a second image includes pixel values of pixels inthe second image and a pixel value gradient of the pixels in the secondimage, where the two images are located in the coordinate system, afirst determining unit 420 configured to determine, using across-correlation measurement model, first coordinate offset accordingto the image information of the two images, where the first coordinateoffset are used to indicate position deviations of to-be-aligned pixelsbetween the two images in the coordinate system, and an alignment unit430 configured to align the two images according to the coordinates ofthe pixels in the first image in the coordinate system and the firstcoordinate offset.

According to this embodiment of the present disclosure, thecross-correlation measurement model is introduced. Because both a colorcross-correlation and a gradient cross-correlation that are betweenimages are considered in the cross-correlation measurement model,compared with an existing image alignment technology based on a SIFTfeature point, the cross-correlation measurement model is more suitablefor alignment between multi-modal multi-spectral images and improvesimage alignment accuracy.

Optionally, in an embodiment, the apparatus 400 may further include asecond obtaining unit (not shown) configured to obtain image informationof a third image, where the image information of the third imageincludes pixel values of pixels in the third image and a pixel valuegradient of the pixels in the third image, the third image is located inthe coordinate system, and the first image and the third image areto-be-aligned original images, a second determining unit (not shown)configured to determine, using the cross-correlation measurement model,a coordinate transformation matrix according to image information of theoriginal images, where the coordinate transformation matrix is used toindicate a spatial position relationship of to-be-aligned pixels betweenthe original images in the coordinate system, a third determining unit(not shown) configured to determine second coordinate offset accordingto the coordinate transformation matrix, where the second coordinateoffset are used to indicate position deviations of the to-be-alignedpixels between the original images in the coordinate system, and afourth determining unit (not shown) configured to obtain the secondimage according to the second coordinate offset and the pixel values ofthe pixels in the third image.

Optionally, in an embodiment, the second determining unit is furtherconfigured to determine the coordinate transformation matrix bycalculating a minimum value of

${{E_{1}(H)} = {\sum\limits_{p}{E_{2}\left( {p,w_{p}} \right)}}},$

where E₂(p,w_(p))=ρ(1−|Φ₁(p,w_(p))|)+τρ(1−|Φ_(∇I)(p,w_(p))|),

${{\Phi_{I}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{3,p} - I_{3,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{3,p} - I_{3,p}^{\prime}}}}},{{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}}}}},{{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$

p=(x_(p),y_(p))^(T), and w_(p)=(u_(p),v_(p))^(T), H indicates thecoordinate transformation matrix, and H meets[u_(p),v_(p),1]^(T)=[x_(p),y_(p),1]^(T)(H−I), I indicates an identitymatrix, p indicates the coordinates of the pixels in the first image inthe coordinate system, x_(p) indicates a horizontal coordinate of p,y_(p) indicates a vertical coordinate of p, w_(p) indicates the secondcoordinate offset, u_(p) indicates a horizontal coordinate of w_(p),v_(p) indicates a vertical coordinate of w_(p), I_(1,p) indicates aone-dimensional column vector including pixel values of pixels in ap-centered image block of the first image, ∇I′_(1,p) indicates aone-dimensional column vector including pixel value gradients of thepixels in the image block, I′_(1,p) indicates a one-dimensional columnvector including pixel means of the pixels in the image block, ∇I′_(1,p)indicates a one-dimensional column vector including pixel value gradientmeans of the pixels in the image block, I_(3,p) indicates aone-dimensional column vector including pixel values of pixels in a(p+w_(p))-centered image block of the third image, ∇I_(3,p) indicates aone-dimensional column vector including pixel value gradients of thepixels in the (p+w_(p))-centered image block, I′_(3,p) indicates aone-dimensional column vector including pixel means of the pixels in the(p+w_(p))-centered image block, ∇I′_(3,p) indicates a one-dimensionalcolumn vector including pixel value gradient means of the pixels in the(p+w_(p))-centered image block, and β and τ are constants, where β isused to control a shape of a function ρ(x), and τ is a weight ofρ(1−|Φ_(∇I)(p,w_(p))| in E₂(p,w_(p)).

Optionally, in an embodiment, the third determining unit is furtherconfigured to determine the second coordinate offset according to aformula [u_(p),v_(p),1]^(T)=[x_(p),y_(p),1]^(T)(H−I).

Optionally, in an embodiment, the fourth determining unit is furtherconfigured to obtain the second image according to a formulaI₂(p)=I₃(p+w_(p)), where I₂(p) indicates a pixel value of the secondimage in p, and I₃(p+w_(p)) indicates a pixel value of the third imagein p+w_(p).

Optionally, in an embodiment, the first determining unit 420 is furtherconfigured to determine the first coordinate offset according to aformula

${{E_{3}\left( w_{p}^{\prime} \right)} = {{\sum\limits_{p}{E_{2}\left( {p,w_{p}^{\prime}} \right)}} + {\lambda_{1}{\sum\limits_{p}{\psi \left( {{\nabla w_{p}^{\prime}}}^{2} \right)}}} + {\lambda_{2}{\sum\limits_{p}{\sum\limits_{q \in {N{(p)}}}{{w_{p}^{\prime} - w_{q}^{\prime}}}}}}}},$

where E₂(p,w′_(p))=ρ(1−|Φ₁(p,w′_(p))|)+τρ(1−|Φ_(∇I)(p,w′_(p))|),

${{\Phi_{I}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},{{\Phi_{\nabla I}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},$

p=(x_(p),y_(p))^(T), w′_(p)=(u′_(p),v′_(p))^(T),

${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$

and ψ(x²)=√{square root over (x²+ε²)}, p indicates the coordinates ofthe pixels in the first image in the coordinate system, x_(p) indicatesthe horizontal coordinate of p, y_(p) indicates the vertical coordinateof p, w′_(p) indicates the first coordinate offset, u′_(p) indicates ahorizontal coordinate of w′_(p), v′_(p) indicates a vertical coordinateof w′_(p), I_(1,p) indicates the one-dimensional column vector includingthe pixel values of the pixels in the p-centered image block of thefirst image, ∇I_(1,p) indicates the one-dimensional column vectorincluding the pixel value gradients of the pixels in the image block,I′_(1,p) indicates the one-dimensional column vector including the pixelmeans of the pixels in the image block, ∇I′_(1,p) indicates theone-dimensional column vector including the pixel value gradient meansof the pixels in the image block, I_(2,p) indicates a one-dimensionalcolumn vector including pixel values of pixels in a (p+w′_(p))-centeredimage block of the second image, ∇I_(2,p) indicates a one-dimensionalcolumn vector including pixel value gradients of the pixels in the(p+w′_(p))-centered image block, I′_(2,p) indicates a one-dimensionalcolumn vector including pixel means of the pixels in the(p+w′_(p))-centered image block, ∇I′_(2,p) indicates a one-dimensionalcolumn vector including pixel value gradient means of the pixels in the(p+w′_(p))-centered image block, λ₁, β, λ₂ and τ are constants, where λ₁and λ₂ are weights of the second term and the third term that are inE₃(w′_(p)), β is used to control the shape of the function ρ(x), and τis the weight of ρ(1−|Φ_(∇I)(p,w′_(p))| in E₂(p,w′_(p)), N(p) indicatesa set including adjacent pixels of a pixel p in the first image, qindicates any pixel in the set, w_(q) indicates a coordinate offsetbetween q and a to-be-aligned pixel of q in the second image, and ε is aconstant.

Optionally, in an embodiment, the cross-correlation measurement model isE₂(p,w_(p))=ρ(1−|Φ₁(p,w_(p))|)+τρ(1−|Φ_(∇I)(p,w_(p))|), where

${{\Phi_{I}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},{{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},$

p=(x_(p),y_(p))^(T), w_(p)=(u_(p),v_(p))^(T), and

${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$

p indicates the coordinates of the pixels in the first image in thecoordinate system, x_(p) indicates the horizontal coordinate of p, y_(p)indicates the vertical coordinate of p, w_(p) indicates the firstcoordinate offset, u_(p) indicates the horizontal coordinate of w_(p),v_(p) indicates the vertical coordinate of w_(p), I_(1,p) indicates theone-dimensional column vector including the pixel values of the pixelsin the p-centered image block of the first image, ∇I_(1,p) indicates theone-dimensional column vector including the pixel value gradients of thepixels in the image block, I′_(1,p) indicates the one-dimensional columnvector including the pixel means of the pixels in the image block,∇I′_(1,p) indicates the one-dimensional column vector including thepixel value gradient means of the pixels in the image block, I_(2,p)indicates a one-dimensional column vector including pixel values ofpixels in a (p+w_(p))-centered image block of the second image, ∇I_(2,p)indicates a one-dimensional column vector including pixel valuegradients of the pixels in the (p+w_(p))-centered image block, I′_(2,p)indicates a one-dimensional column vector including pixel means of thepixels in the (p+w_(p))-centered image block, ∇I′_(2,p) indicates aone-dimensional column vector including pixel value gradient means ofthe pixels in the (p+w_(p))-centered image block, and β is a weight andis used to control the shape of the function ρ(x).

FIG. 5 is a schematic block diagram of an image alignment apparatusaccording to an embodiment of the present disclosure. The apparatus 500in FIG. 5 includes a memory 510 configured to store a program, and aprocessor 520 configured to execute the program. When the program isexecuted, the processor 520 is configured to obtain image information oftwo to-be-aligned images, where image information of a first imageincludes coordinates of pixels in the first image in a selectedcoordinate system, pixel values of the pixels in the first image, and apixel value gradient of the pixels in the first image, and imageinformation of a second image includes pixel values of pixels in thesecond image and a pixel value gradient of the pixels in the secondimage, where the two images are located in the coordinate system,determine, using a cross-correlation measurement model, first coordinateoffset according to the image information of the two images, where thefirst coordinate offset are used to indicate position deviations ofto-be-aligned pixels between the two images in the coordinate system,and align the two images according to the coordinates of the pixels inthe first image in the coordinate system and the first coordinateoffset.

According to this embodiment of the present disclosure, thecross-correlation measurement model is introduced. Because both a colorcross-correlation and a gradient cross-correlation that are betweenimages are considered in the cross-correlation measurement model,compared with an existing image alignment technology based on a SIFTfeature point, the cross-correlation measurement model is more suitablefor alignment between multi-modal multi-spectral images and improvesimage alignment accuracy.

Optionally, in an embodiment, the processor 520 may be furtherconfigured to obtain image information of a third image, where the imageinformation of the third image includes pixel values of pixels in thethird image and a pixel value gradient of the pixels in the third image,the third image is located in the coordinate system, and the first imageand the third image are to-be-aligned original images, determine, usingthe cross-correlation measurement model, a coordinate transformationmatrix according to image information of the original images, where thecoordinate transformation matrix is used to indicate a spatial positionrelationship of to-be-aligned pixels between the original images in thecoordinate system, determine second coordinate offset according to thecoordinate transformation matrix, where the second coordinate offset areused to indicate position deviations of the to-be-aligned pixels betweenthe original images in the coordinate system, and obtain the secondimage according to the second coordinate offset and the pixel values ofthe pixels in the third image.

Optionally, in an embodiment, the processor 520 is further configured todetermine the coordinate transformation matrix by calculating a minimumvalue of

${{E_{1}(H)} = {\sum\limits_{p}\; {E_{2}\left( {p,w_{p}} \right)}}},$

where E₂(p,w_(p))=ρ(1−|Φ₁(p,w_(p))|)+τρ(1−|Φ_(∇I)(p,w_(p))|),

${{\Phi_{I}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{3,p} - I_{3,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{3,p} - I_{3,p}^{\prime}}}}},{{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}}}}},{{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$

p=(x_(p),y_(p))^(T), and w_(p)=(u_(p),v_(p))^(T), H indicates thecoordinate transformation matrix, and H meets[u_(p),v_(p),1]^(T)=[x_(p),y_(p),1]^(T)(H−I), I indicates an identitymatrix, p indicates the coordinates of the pixels in the first image inthe coordinate system, x_(p) indicates a horizontal coordinate of p,y_(p) indicates a vertical coordinate of p, w_(p) indicates the secondcoordinate offset, u_(p) ndicates a horizontal coordinate of w_(p),v_(p) indicates a vertical coordinate of w_(p). I_(1,p) indicates aone-dimensional column vector including pixel values of pixels in ap-centered image block of the first image, ∇I_(1,p) indicates aone-dimensional column vector including pixel value gradients of thepixels in the image block, I′_(1,p) indicates a one-dimensional columnvector including pixel means of the pixels in the image block, ∇I′_(1,p)indicates a one-dimensional column vector including pixel value gradientmeans of the pixels in the image block, I_(3,p) indicates aone-dimensional column vector including pixel values of pixels in a(p+w_(p))-centered image block of the third image, ∇I_(3,p) indicates aone-dimensional column vector including pixel value gradients of thepixels in the (p+w_(p))-centered image block, I′_(3,p) indicates aone-dimensional column vector including pixel means of the pixels in the(p+w_(p))-centered image block, ∇I′_(3,p) indicates a one-dimensionalcolumn vector including pixel value gradient means of the pixels in the(p+w_(p))-centered image block, and β and τ are constants, where β isused to control a shape of a function ρ(x), and τ is a weight ofρ(1−|Φ_(∇I)(p,w_(p))| in E₂(p,w_(p)).

Optionally, in an embodiment, the processor 520 is further configured todetermine the second coordinate offset according to a formula[u_(p),v_(p),1]^(T)=[x_(p),y_(p),1]^(T)(H−I).

Optionally, in an embodiment, the processor 520 is further configured toobtain the second image according to a formula I₂(p)=I₃(p+w_(p)), whereI₂(p) indicates a pixel value of the second image in p, and I₃(p+w_(p))indicates a pixel value of the third image in p+w_(p).

Optionally, in an embodiment, the processor 520 is further configured todetermine the first coordinate offset according to a formula

${{E_{3}\left( w_{p}^{\prime} \right)} = {{\sum\limits_{p}\; {E_{2}\left( {p,w_{p}^{\prime}} \right)}} + {\lambda_{1}{\sum\limits_{p}\; {\psi \left( {{\nabla w_{p}^{\prime}}}^{2} \right)}}} + {\lambda_{2}{\sum\limits_{p}\; {\sum\limits_{q \in {N{(p)}}}\; {{w_{p}^{\prime} - w_{q}^{\prime}}}}}}}},$

where E₂(p,w′_(p))=ρ(1−|Φ₁(p,w′_(p))|)+τρ(1−|Φ_(∇I)(p,w′_(p))|),

${{\Phi_{I}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},{{\Phi_{\nabla I}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},$

p=(x_(p),y_(p))^(T), w′_(p)=(u′_(p),v′_(p))^(T),

${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$

and ψ(x²)=√{square root over (x²+ε²)}, p indicates the coordinates ofthe pixels in the first image in the coordinate system, x_(p) indicatesthe horizontal coordinate of p, y_(p) indicates the vertical coordinateof p, w′_(p) indicates the first coordinate offset, u′_(p) indicates ahorizontal coordinate of w′_(p), v′_(p) indicates a vertical coordinateof w′_(p), I_(1,p) indicates the one-dimensional column vector includingthe pixel values of the pixels in the p-centered image block of thefirst image, ∇I_(1,p) indicates the one-dimensional column vectorincluding the pixel value gradients of the pixels in the image block,I′_(1,p) indicates the one-dimensional column vector including the pixelmeans of the pixels in the image block, ∇I′_(1,p) indicates theone-dimensional column vector including the pixel value gradient meansof the pixels in the image block, I_(2,p) indicates a one-dimensionalcolumn vector including pixel values of pixels in a (p+w′_(p))-centeredimage block of the second image, ∇I_(2,p) indicates a one-dimensionalcolumn vector including pixel value gradients of the pixels in the(p+w_(p))-centered image block, I′_(2,p) indicates a one-dimensionalcolumn vector including pixel means of the pixels in the(p+w_(p))-centered image block, ∇I′_(2,p) indicates a one-dimensionalcolumn vector including pixel value gradient means of the pixels in the(p+w′_(p))-centered image block, λ₁, β, λ₂ and τ are constants, where λ₁and λ₂ are weights of the second term and the third term that are inE₃(w′_(p)), β is used to control the shape of the function ρ(x), and τis the weight of ρ(1−|Φ_(∇I)(p,w′_(p))| in E₂(p,w′_(p)), N(p) indicatesa set including adjacent pixels of a pixel p in the first image, qindicates any pixel in the set, w_(q) indicates a coordinate offsetbetween q and a to-be-aligned pixel of q in the second image, and ε is aconstant.

Optionally, in an embodiment, the cross-correlation measurement model isE₂(p,w_(p))=ρ(1−|Φ₁(p,w_(p))|)+τρ(1−|Φ_(∇I)(p,w_(p))|), where

${{\Phi_{I}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},{{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},$

p=(x_(p),y_(p))^(T), w_(p)=(u_(p),v_(p))^(T), and

${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$

p indicates the coordinates of the pixels in the first image in thecoordinate system, x_(p) indicates the horizontal coordinate of p,y_(p)indicates the vertical coordinate of p, w_(p) indicates the firstcoordinate offset, u_(p) indicates the horizontal coordinate of w_(p),v_(p) indicates the vertical coordinate of w_(p), I_(1,p) indicates theone-dimensional column vector including the pixel values of the pixelsin the p-centered image block of the first image, ∇I_(1,p) indicates theone-dimensional column vector including the pixel value gradients of thepixels in the image block, I′_(1,p) indicates the one-dimensional columnvector including the pixel means of the pixels in the image block,∇I′_(1,p) indicates the one-dimensional column vector including thepixel value gradient means of the pixels in the image block, I_(2,p)indicates a one-dimensional column vector including pixel values ofpixels in a (p+w_(p))-centered image block of the second image, ∇I_(2,p)indicates a one-dimensional column vector including pixel valuegradients of the pixels in the (p+w_(p))-centered image block, I′_(2,p)indicates a one-dimensional column vector including pixel means of thepixels in the (p+w_(p))-centered image block, ∇I′_(2,p) indicates aone-dimensional column vector including pixel value gradient means ofthe pixels in the (p+w_(p))-centered image block, and β is a weight andis used to control the shape of the function ρ(x).

It should be understood that, the term “and/or” in this embodiment ofthe present disclosure describes only an association relationship fordescribing associated objects and represents that three relationshipsmay exist. For example, A and/or B may represent the following threecases, only A exists, both A and B exist, and only B exists. Inaddition, the character “/” in this specification generally indicates an“or” relationship between the associated obj ects.

A person of ordinary skill in the art may be aware that, in combinationwith the examples described in the embodiments disclosed in thisspecification, units and algorithm steps may be implemented byelectronic hardware, computer software, or a combination thereof. Toclearly describe the interchangeability between the hardware and thesoftware, the foregoing has generally described compositions and stepsof each example according to functions. Whether the functions areperformed by hardware or software depends on particular applications anddesign constraint conditions of the technical solutions. A personskilled in the art may use different methods to implement the describedfunctions for each particular application, but it should not beconsidered that the implementation goes beyond the scope of the presentdisclosure.

It may be clearly understood by a person skilled in the art that, forthe purpose of convenient and brief description, for a detailed workingprocess of the foregoing system, apparatus, and unit, reference may bemade to a corresponding process in the foregoing method embodiments, anddetails are not described herein again.

In the several embodiments provided in this application, it should beunderstood that the disclosed system, apparatus, and method may beimplemented in other manners. For example, the described apparatusembodiment is merely an example. For example, the unit division ismerely logical function division and may be other division in actualimplementation. For example, a plurality of units or components may becombined or integrated into another system, or some features may beignored or not performed. In addition, the displayed or discussed mutualcouplings or direct couplings or communication connections may beimplemented through some interfaces. The indirect couplings orcommunication connections between the apparatuses or units may beimplemented in electronic, mechanical, or other forms.

The units described as separate parts may or may not be physicallyseparate, and parts displayed as units may or may not be physical units,may be located in one position, or may be distributed on a plurality ofnetwork units. A part or all of the units may be selected according toactual needs to achieve the objectives of the solutions of theembodiments of the present disclosure.

In addition, functional units in the embodiments of the presentdisclosure may be integrated into one processing unit, or each of theunits may exist alone physically, or two or more units are integratedinto one unit. The integrated unit may be implemented in a form ofhardware, or may be implemented in a form of a software functional unit.

When the integrated unit is implemented in the form of a softwarefunctional unit and sold or used as an independent product, theintegrated unit may be stored in a computer-readable storage medium.Based on such an understanding, the technical solutions of the presentdisclosure essentially, or the part contributing to the prior art, orall or a part of the technical solutions may be implemented in the formof a software product. The software product is stored in a storagemedium and includes several instructions for instructing a computerdevice (which may be a personal computer, a server, or a network device)to perform all or a part of the steps of the methods described in theembodiments of the present disclosure. The foregoing storage mediumincludes any medium that can store program code, such as a universalserial bus (USB) flash drive, a removable hard disk, a read-only memory(ROM), a random access memory (RAM), a magnetic disk, or an opticaldisc.

The foregoing descriptions are merely specific embodiments of thepresent disclosure, but are not intended to limit the protection scopeof the present disclosure. Any modification or replacement readilyfigured out by a person skilled in the art within the technical scopedisclosed in the present disclosure shall fall within the protectionscope of the present disclosure. Therefore, the protection scope of thepresent disclosure shall be subject to the protection scope of theclaims.

What is claimed is:
 1. An image alignment method, comprising: obtainingimage information of two to-be-aligned images, wherein image informationof a first image comprises coordinates of pixels in the first image in aselected coordinate system, pixel values of the pixels in the firstimage, and a pixel value gradient of the pixels in the first image,wherein image information of a second image comprises pixel values ofpixels in the second image and a pixel value gradient of the pixels inthe second image, and wherein the two images are located in the selectedcoordinate system; determining, using a cross-correlation measurementmodel, first coordinate offset according to the image information of thetwo images, wherein the first coordinate offset indicate positiondeviations of to-be-aligned pixels between the two images in theselected coordinate system; and aligning the two images according to thecoordinates of the pixels in the first image in the selected coordinatesystem and the first coordinate offset.
 2. The method according to claim1, wherein before determining the first coordinate offset, the methodfurther comprises: obtaining image information of a third image, whereinthe image information of the third image comprises pixel values ofpixels in the third image and a pixel value gradient of the pixels inthe third image, wherein the third image is located in the selectedcoordinate system, and wherein the first image and the third image areto-be-aligned original images; determining, using the cross-correlationmeasurement model, a coordinate transformation matrix according to imageinformation of the to-be-aligned original images, wherein the coordinatetransformation matrix indicates a spatial position relationship ofto-be-aligned pixels between the to-be-aligned original images in theselected coordinate system; determining second coordinate offsetaccording to the coordinate transformation matrix, wherein the secondcoordinate offset are used to indicate position deviations of theto-be-aligned pixels between the to-be-aligned original images in theselected coordinate system; and obtaining the second image according tothe second coordinate offset and the pixel values of the pixels in thethird image.
 3. The method according to claim 2, wherein determining thecoordinate transformation matrix comprises determining the coordinatetransformation matrix by calculating a minimum value of${{E_{1}(H)} = {\sum\limits_{p}\; {E_{2}\left( {p,w_{p}} \right)}}},$wherein E₂(p,w_(p))=ρ(1−|Φ₁(p,w_(p))|)+τρ(1−|Φ_(∇I)(p,w_(p))|), wherein${{\Phi_{I}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{3,p} - I_{3,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{3,p} - I_{3,p}^{\prime}}}}},$wherein${{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}}}}},$wherein${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$wherein p=(x_(p),y_(p))^(T), wherein w_(p)=(u_(p),v_(p))^(T), Hindicates the coordinate transformation matrix, and meets[u_(p),v_(p),1]^(T)=[x_(p),y_(p),1]^(T)(H−I), wherein i indicates anidentity matrix, wherein p indicates the coordinates of the pixels inthe first image in the selected coordinate system, wherein x_(p)indicates a horizontal coordinate of p, wherein y_(p) indicates avertical coordinate of p, wherein w_(p) indicates the second coordinateoffset, wherein u_(p) indicates a horizontal coordinate of w_(p),wherein v_(p) indicates a vertical coordinate of w_(p), wherein I_(1,p)indicates a one-dimensional column vector comprising pixel values ofpixels in a p-centered image block of the first image, wherein ∇I_(1,p)indicates a one-dimensional column vector comprising pixel valuegradients of the pixels in the image block, wherein I′_(1,p) indicates aone-dimensional column vector comprising pixel means of the pixels inthe image block, wherein ∇I′_(1,p) indicates a one-dimensional columnvector comprising pixel value gradient means of the pixels in the imageblock, wherein I_(3,p) indicates a one-dimensional column vectorcomprising pixel values of pixels in a (p+w_(p))-centered image block ofthe third image, wherein ∇I_(3,p) indicates a one-dimensional columnvector comprising pixel value gradients of the pixels in the(p+w_(p))-centered image block, wherein I′_(3,p)) indicates aone-dimensional column vector comprising pixel means of the pixels inthe (p+w_(p))-centered image block, wherein ∇I′_(3,p) indicates aone-dimensional column vector comprising pixel value gradient means ofthe pixels in the (p+w_(p))-centered image block, wherein β and τ areconstants, wherein β controls a shape of a function ρ(x), and wherein τis a weight of ρ(1−|Φ_(∇I)(p,w_(p))| in E₂(p,w_(p)).
 4. The methodaccording to claim 3, wherein determining the second coordinate offsetcomprises determining the second coordinate offset according to aformula [u_(p),v_(p),1]^(T)=[x_(p),y_(p),1]^(T)(H−I).
 5. The methodaccording to claim 4, wherein obtaining the second image comprisesobtaining the second image according to a formula I₂(p)=I₃(p+w_(p)),wherein I₂(p) indicates a pixel value of the second image in p, andwherein I₃(p+w_(p)) indicates a pixel value of the third image inp+w_(p).
 6. The method according to claim 1, wherein determining thefirst coordinate offset comprises determining the first coordinateoffset according to a formula${{E_{3}\left( w_{p}^{\prime} \right)} = {{\sum\limits_{p}\; {E_{2}\left( {p,w_{p}^{\prime}} \right)}} + {\lambda_{1}{\sum\limits_{p}\; {\psi \left( {{\nabla w_{p}^{\prime}}}^{2} \right)}}} + {\lambda_{2}{\sum\limits_{p}\; {\sum\limits_{q \in {N{(p)}}}\; {{w_{p}^{\prime} - w_{q}^{\prime}}}}}}}},$wherein E₂(p,w′_(p))=ρ(1−|Φ₁(p,w′_(p))|)+τρ(1−|Φ_(∇I)(p,w′_(p))|),wherein${{\Phi_{I}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},$wherein${{\Phi_{\nabla I}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},$wherein p=(x_(p),y_(p))^(T), wherein w′_(p)=(u′_(p),v′_(p))^(T), wherein${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$wherein ψ(x²)=√{square root over (x²+ε²)}, wherein p indicates thecoordinates of the pixels in the first image in the selected coordinatesystem, wherein x_(p) indicates a horizontal coordinate of p, whereiny_(p) indicates a vertical coordinate of p, wherein w′_(p) indicates thefirst coordinate offset, wherein u′_(p) indicates a horizontalcoordinate of w′_(p), wherein v′_(p) indicates a vertical coordinate ofw′_(p), wherein I_(1,p) indicates a one-dimensional column vectorcomprising pixel values of pixels in a p-centered image block of thefirst image, wherein ∇I_(1,p) indicates a one-dimensional column vectorcomprising pixel value gradients of the pixels in the image block,wherein I′_(1,p) indicates a one-dimensional column vector comprisingpixel means of the pixels in the image block, wherein ∇I′I_(1,p)indicates a one-dimensional column vector comprising pixel valuegradient means of the pixels in the image block, wherein I_(2,p)indicates a one-dimensional column vector comprising pixel values ofpixels in a (p+w′_(p))-centered image block of the second image, wherein∇I_(2,p) indicates a one-dimensional column vector comprising pixelvalue gradients of the pixels in the (p+w′_(p))-centered image block,wherein I′_(2,p) indicates a one-dimensional column vector comprisingpixel means of the pixels in the (p+w′_(p))-centered image block,wherein ∇I′_(w,p) indicates a one-dimensional column vector comprisingpixel value gradient means of the pixels in the (p+w′_(p))-centeredimage block, wherein λ₁, β, λ₂ and τ are constants, wherein λ₁ and λ₂are weights of the second term and the third term in E₃(w′_(p)), whereinβ controls the shape of the function ρ(x), wherein τ is the weight ofρ(1−|Φ_(∇I)(p,w′_(p))| in E₂(p,w′_(p)), wherein N(p) indicates a setcomprising adjacent pixels of a pixel p in the first image, wherein qindicates any pixel in the set, wherein w_(q) indicates a coordinateoffset between q and a to-be-aligned pixel of q in the second image, andwherein ε is a constant.
 7. The method according to(claim(1, wherein thecross-correlation measurement model comprisesE₂(p,w_(p))=ρ(1−|Φ₁(p,w_(p))|)+τρ(1−|Φ_(∇I)(p,w_(p))|), wherein${{\Phi_{I}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},$wherein${{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},$p=(x_(p),y_(p))^(T), wherein w_(p)=(u_(p),v_(p))^(T), wherein${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$wherein p indicates the coordinates of the pixels in the first image inthe selected coordinate system, wherein x_(p) indicates a horizontalcoordinate of p, wherein y_(p) indicates a vertical coordinate of p,wherein w_(p) indicates the first coordinate offset, wherein u_(p)indicates a horizontal coordinate of w_(p), wherein v_(p) indicates avertical coordinate of w_(p), wherein I_(1,p) indicates aone-dimensional column vector comprising pixel values of pixels in thep-centered image block of the first image, wherein ∇I_(1,p) indicates aone-dimensional column vector comprising pixel value gradients of thepixels in the image block, wherein I′_(1,p) indicates a one-dimensionalcolumn vector comprising the pixel means of the pixels in the imageblock, wherein ∇I′_(1,p) indicates a one-dimensional column vectorcomprising pixel value gradient means of the pixels in the image block,wherein I_(2,p) indicates a one-dimensional column vector comprisingpixel values of pixels in a (p+w_(p))-centered image block of the secondimage, wherein ∇I′_(2,p) indicates a one-dimensional column vectorcomprising pixel value gradients of the pixels in the (p+w_(p))-centeredimage block, wherein I′_(2,p) indicates a one-dimensional column vectorcomprising pixel means of the pixels in the (p+w_(p))-centered imageblock, wherein ∇I′_(2,p) indicates a one-dimensional column vectorcomprising pixel value gradient means of the pixels in the(p+w_(p))-centered image block, and wherein β is a weight and controlsthe shape of the function ρ(x).
 8. An image alignment apparatus,comprising: a memory configured to store instructions; and a processorcoupled to the memory and configured to execute the instructions to:obtain image information of two to-be-aligned images, wherein imageinformation of a first image comprises coordinates of pixels in thefirst image in a selected coordinate system, pixel values of the pixelsin the first image, and a pixel value gradient of the pixels in thefirst image, wherein image information of a second image comprises pixelvalues of pixels in the second image and a pixel value gradient of thepixels in the second image, and wherein the two images are located inthe selected coordinate system; determine, using a cross-correlationmeasurement model, a first coordinate offset according to the imageinformation of the two images, wherein the first coordinate offsetindicates position deviations of to-be-aligned pixels between the twoimages in the selected coordinate system; and align the two imagesaccording to the coordinates of the pixels in the first image in theselected coordinate system and the first coordinate offset.
 9. Theapparatus according to claim 8, wherein the processor is furtherconfigured to execute the instructions to: obtain image information of athird image, wherein the image information of the third image comprisespixel values of pixels in the third image and a pixel value gradient ofthe pixels in the third image, wherein the third image is located in theselected coordinate system, and wherein the first image and the thirdimage are to-be-aligned original images; determine, using thecross-correlation measurement model, a coordinate transformation matrixaccording to image information of the to-be-aligned original images,wherein the coordinate transformation matrix is used to indicate aspatial position relationship of to-be-aligned pixels between theto-be-aligned original images in the selected coordinate system;determine second coordinate offset according to the coordinatetransformation matrix, wherein the second coordinate offset indicatesposition deviations of the to-be-aligned pixels between theto-be-aligned original images in the selected coordinate system; andobtain the second image according to the second coordinate offset andthe pixel values of the pixels in the third image.
 10. The apparatusaccording to claim 9, wherein the processor is further configured toexecute the instructions to determine the coordinate transformationmatrix by calculating a minimum value of${{E_{1}(H)} = {\sum\limits_{p}\; {E_{2}\left( {p,w_{p}} \right)}}},$wherein E₂(p,w_(p))=ρ(1−|Φ₁(p,w_(p))|)+τρ(1−|Φ_(∇I)(p,w_(p))|), wherein${{\Phi_{I}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{3,p} - I_{3,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{3,p} - I_{3,p}^{\prime}}}}},$wherein${{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{3,p}} - {\nabla I_{3,p}^{\prime}}}}}},$wherein${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$wherein p=(x_(p),y_(p))^(T), wherein w_(p)=(u_(p),v_(p))^(T), wherein Hindicates the coordinate transformation matrix and meets[u_(p),v_(p),1]^(T)=[x_(p),y_(p),1]^(T)(H−I), wherein I indicates anidentity matrix, wherein p indicates the coordinates of the pixels inthe first image in the selected coordinate system, wherein x_(p)indicates a horizontal coordinate of p, wherein y_(p) indicates avertical coordinate of p, wherein w_(p) indicates the second coordinateoffset, wherein u_(p) indicates a horizontal coordinate of w_(p),wherein v_(p) indicates a vertical coordinate of w_(p), wherein I_(1,p)indicates a one-dimensional column vector comprising pixel values ofpixels in a p-centered image block of the first image, wherein ∇I_(1,p)indicates a one-dimensional column vector comprising pixel valuegradients of the pixels in the image block, wherein I′_(1,p) indicates aone-dimensional column vector comprising pixel means of the pixels inthe image block, wherein ∇I′_(1,p) indicates a one-dimensional columnvector comprising pixel value gradient means of the pixels in the imageblock, wherein I_(3,p) indicates a one-dimensional column vectorcomprising pixel values of pixels in a (p+w_(p))-centered image block ofthe third image, wherein ∇I_(3,p) indicates a one-dimensional columnvector comprising pixel value gradients of the pixels in the(p+w_(p))-centered image block, wherein I′_(3,p) indicates aone-dimensional column vector comprising pixel means of the pixels inthe (p+w_(p))-centered image block, wherein ∇I′_(3,p) indicates aone-dimensional column vector comprising pixel value gradient means ofthe pixels in the (p+w_(p))-centered image block, wherein β and τ areconstants, wherein β controls a shape of a function ρ(x), and wherein τis a weight of ρ(1−|Φ_(∇I)(p,w_(p))| in E₂(p,w_(p)).
 11. The apparatusaccording to claim 10, wherein the processor is further configured toexecute the instructions to determine the second coordinate offsetaccording to a formula [u_(p),v_(p),1]^(T)=[x_(p),y_(p),1]^(T)(H−I). 12.The apparatus according to claim 11, wherein the processor is furtherconfigured to execute the instructions to obtain the second imageaccording to a formula I₂(p)=I₃(p+w_(p)), wherein I₂(p) indicates apixel value of the second image in p, and wherein I₃(p+w_(p)) indicatesa pixel value of the third image in p+w_(p).
 13. The apparatus accordingto claim 8, wherein the processor is further configured to execute theinstructions to determine the first coordinate offset according to aformula${{E_{3}\left( w_{p}^{\prime} \right)} = {{\sum\limits_{p}\; {E_{2}\left( {p,w_{p}^{\prime}} \right)}} + {\lambda_{1}{\sum\limits_{p}\; {\psi \left( {{\nabla w_{p}^{\prime}}}^{2} \right)}}} + {\lambda_{2}{\sum\limits_{p}\; {\sum\limits_{q \in {N{(p)}}}\; {{w_{p}^{\prime} - w_{q}^{\prime}}}}}}}},$wherein E₂(p,w′_(p))=ρ(1−|Φ₁(p,w′_(p))|)+τρ(1−|Φ_(∇I)(p,w′_(p))|),wherein${{\Phi_{I}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},$wherein${{\Phi_{\nabla I}\left( {p,w_{p}^{\prime}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},$wherein p=(x_(p),y_(p))^(T), wherein w′_(p)=(u′_(p),v′_(p))^(T), wherein${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$wherein ψ(x²)=√{square root over (x²+ε²)}, wherein p indicates thecoordinates of the pixels in the first image in the selected coordinatesystem, wherein x_(p) indicates a horizontal coordinate of p, whereiny_(p) indicates a vertical coordinate of p, wherein w′_(p) indicates thefirst coordinate offset, wherein u′_(p) indicates a horizontalcoordinate of w′_(p), wherein v′_(p) indicates a vertical coordinate ofw′_(p), wherein I_(1,p) indicates a one-dimensional column vectorcomprising pixel values of pixels in a p-centered image block of thefirst image, wherein ∇I_(1,p) indicates a one-dimensional column vectorcomprising pixel value gradients of the pixels in the image block,wherein I′_(1,p) indicates a one-dimensional column vector comprisingpixel means of the pixels in the image block, wherein ∇I′_(1,p)indicates a one-dimensional column vector comprising a pixel valuegradient means of the pixels in the image block, wherein I_(2,p)indicates a one-dimensional column vector comprising pixel values ofpixels in a (p+w′_(p))-centered image block of the second image, wherein∇I_(2,p) indicates a one-dimensional column vector comprising pixelvalue gradients of the pixels in the (p+w′_(p))-centered image block,wherein I′_(2,p) indicates a one-dimensional column vector comprisingpixel means of the pixels in the (p+w′_(p))-centered image block,wherein ∇I′_(2,p) indicates a one-dimensional column vector comprisingpixel value gradient means of the pixels in the (p+w′_(p))-centeredimage block, wherein λ₁, β, λ₂ and τ are constants, wherein λ₁ and λ₂are weights of the second term and the third term in E₃(w′_(p)), whereinβ controls the shape of the function ρ(x), wherein τ is the weight ofρ(1−|Φ_(∇I)(p,w′_(p))| E₂(p,w′_(p)), wherein N(p) indicates a setcomprising adjacent pixels of a pixel p in the first image, wherein qindicates any pixel in the set, wherein w_(q) indicates a coordinateoffset between q and a to-be-aligned pixel of q in the second image; andwherein ε is a constant.
 14. The apparatus according to claim 8, whereinthe cross-correlation measurement model comprisesE₂(p,w_(p))=ρ(1−|Φ₁(p,w_(p))|)+τρ(1−|Φ_(∇I)(p,w_(p))|), wherein${{\Phi_{I}\left( {p,w_{p}} \right)} = \frac{\left( {I_{1,p} - I_{1,p}^{\prime}} \right)^{T}\left( {I_{2,p} - I_{2,p}^{\prime}} \right)}{{{I_{1,p} - I_{1,p}^{\prime}}}{{I_{2,p} - I_{2,p}^{\prime}}}}},$wherein${{\Phi_{\nabla I}\left( {p,w_{p}} \right)} = \frac{\left( {{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}} \right)^{T}\left( {{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}} \right)}{{{{\nabla I_{1,p}} - {\nabla I_{1,p}^{\prime}}}}{{{\nabla I_{2,p}} - {\nabla I_{2,p}^{\prime}}}}}},$wherein p=(x_(p),y_(p))^(T), wherein w_(p)=(u_(p),v_(p))^(T), wherein${{\rho (x)} = {{- \frac{1}{\beta}}{\log \left( {e^{{- \beta}{x}} + e^{- {\beta {({2 - {x}})}}}} \right)}}},$wherein p indicates the coordinates of the pixels in the first image inthe selected coordinate system, wherein s_(p) indicates a horizontalcoordinate of p, wherein y_(p) indicates a vertical coordinate of p,wherein w_(p) indicates the first coordinate offset, wherein u_(p)indicates a horizontal coordinate of w_(p), wherein v_(p) indicates avertical coordinate of w_(p), wherein I_(1,p) indicates aone-dimensional column vector comprising pixel values of pixels in thep-centered image block of the first image, wherein ∇I_(1,p) indicates aone-dimensional column vector comprising pixel value gradients of thepixels in the image block, wherein I′_(1,p) indicates a one-dimensionalcolumn vector comprising the pixel means of the pixels in the imageblock, wherein ∇I′_(1,p) indicates a one-dimensional column vectorcomprising pixel value gradient means of the pixels in the image block,wherein I_(2,p) indicates a one-dimensional column vector comprisingpixel values of pixels in a (p+w_(p))-centered image block of the secondimage, wherein ∇I_(2,p) indicates a one-dimensional column vectorcomprising pixel value gradients of the pixels in the (p+w_(p))-centeredimage block, wherein I′_(2,p) indicates a one-dimensional column vectorcomprising pixel means of the pixels in the (p+w_(p))-centered imageblock, wherein ∇I′_(2,p) indicates a one-dimensional column vectorcomprising pixel value gradient means of the pixels in the(p+w_(p))-centered image block, and wherein β is a weight and controlsthe shape of the function ρ(x).